1. Field of the Invention
The present invention relates to a multimedia browser, and more particularly to a structure and method for browsing the multimedia with defined priorities and/or weights of multimedia data segments and semantic elements.
2. Description of the Related Art
Contents of a multimedia program are typically divided into a structural content and a semantic content.
The structural content represents how a multimedia program is physically and logically organized. For example, for a news program, a structural content may be organized by an initial title portion, a headline news portion, and a main news portion including news items. For a drama or a movie, a structural content may be organized according to video sections of the drama or movie story as developed with respect to time. Here, the headline news portion of a news program deals with a plurality of news items and may be further divided into sections corresponding to each news item. Similarly, the main news portion may further be divided into sections corresponding to news items and each news item may be divided into anchor shots and reporter shots.
As described above, division of a multimedia program into several portions is called segmentation and the divided portions are called segments. That is, the structural content is generally described with segments. In the above example, if the news program is logically divided, the news program would have a tree structure representing an example of the structural content.
FIG. 1 shows an example structure information of an audio/video stream. As shown in FIG. 1, the audio/video stream may be represented by a group of scene segments, and the scene segments may be described with a tree structure of shot segments. The structure information of segments is not fixed to a single structure but can have various other structure according to different viewpoints or criteria for the segmentation. Namely, different segments would result from segmentation with reference to audio information, visual information, or logical information. However, any such information is called structure content or structure information and each segment is called a structure element.
On the other hand, the semantic content is information representing human perception and an object-oriented description of a multimedia program. The semantic content may include abstract concepts such as fear, sadness, and happiness; objects such as characters, places, or things appearing in the multimedia program; events which occur in the multimedia program; and relation connecting the objects and events. For example, in a drama, the actors, the character roles, and events describing a plot would be the semantic content; and each actor, character role, and event would be the semantic elements. If the semantic elements are defined by relations among the elements, a semantic structure would be generated.
FIG. 2 is an example semantic information for a motion picture such as a drama or a movie. As shown, the story of Pinocchio is represented by a semantic oriented description with characters such as Geppetto and Pinocchio, and relations and roles between Geppetto and Pinocchio.
The semantic elements and structure information as described above has relations. The relation may be represented by xe2x80x98a semantic element described in a segment of a selected programxe2x80x99 or in reverse, xe2x80x98a segment including a particular semantic element.xe2x80x99 There are other various methods and structures for describing the relations, depending on the way the semantic elements and the structural information are formed. For the examples shown in FIGS. 1 and 2, there may be information indicating the segments in which the character Pinocchio appears or there may be information on a segment indicating the objects and events shown in the segment such as Pinocchio.
Using the above link information, i.e. the relations, users may easily browse a multimedia section or content as desired. Since a multimedia data includes the link information of the structure information and semantic information, users can search for a specific segment of a video or audio to allow a more convenient browsing. That is, users can specifically browse portions of a video rather than watching the whole video, and effectively understand the contents of the whole video in a short period of time by browsing a summary of the video content.
For such browsing, each semantic element includes information connecting the semantic elements to each corresponding segment, or each segment includes information connecting the segment to each corresponding semantic element. Moreover, both are organized into a data structure including a priority or a key word for each segment or semantic element. Accordingly, a multimedia has video segments and semantic elements which allow browsing by connecting semantic element information based on video segments or by connecting video segments based on semantic elements.
Furthermore, a video is divided into units of a shot and multimedia browsing may be carried out based upon a shot segmentation and a shot clustering by which each shot is divided and grouped into units of a scene. Namely, using the shot information and scene information, key frames which represent a scene can be set for a video browsing and searching. Therefore, users may selectively browse or search a specific portion of a video stream. For example, a video browsing method has been proposed using a hierarchically decomposed or a scene transition graph describing a temporal relationship among scenes.
As described above, to allow a browsing of a specific portion of a video, typical video indexing systems establish a data structure based on each semantic characteristic element. That is, a browsing based on appearances of characters or on event sections is provided. However, in such multimedia browser based on content, a significant amount of overlap occurs to describe each semantic elements of each segments or each segments of semantic elements.
Accordingly, an object of the present invention is to solve at least the problems and disadvantages of the related art.
Therefore, an object of the present invention is to provide a data structure and method for a more efficient browsing of data.
Another object of the present invention is to provide a data structure and method with reduced overlapping in description of data.
A still another object of the present invention is to provide a data structure to describe priority and/or weight (priority/weight) information based on semantic elements which is described with link information between structure elements and semantic elements, and a structure for describing the priority/weight information in which qualitative information representing how precisely a structure element describes a semantic element is quantitatively described in the link information.
A further object of the present invention is to provide a structure for describing priority/weight information based on segments in which information representing priority/weight of semantic elements is differentiated depending on the semantic elements.
A still further object of the present invention is to provide a structure for defining and describing individual priority/weight of segments and semantic elements of a multimedia in which link information includes both priority/weight information based on the semantic elements and priority/weight information based on the segments to describe individual priority/weight for corresponding elements.
Additional advantages, objects, and features of the invention will be set forth in part in the description which follows and in part will become apparent to those having ordinary skill in the art upon examination of the following or may be learned from practice of the invention. The objects and advantages of the invention may be realized and attained as particularly pointed out in the appended claims.
To achieve the objects and in accordance with the purposes of the invention, as embodied and broadly described herein, a multimedia data structure includes link information for connecting a semantic element structure and a segment information structure, where the link information includes priority/weight information between semantic elements and segments as an attribute, when a multimedia data includes the semantic element structure describing a content of a multimedia and the segment information structure of the multimedia for browsing the multimedia based on content. The priority/weight information may further includes information representing how precise the priority/weight is described.
In the above embodiment, the priority/weight information may be priority/weight information based on the semantic elements representing the priority/weight of semantic information related to a segment. The priority/weight information may be priority/weight information of structure elements which varies depending on the semantic elements. The priority/weight information may also be priority information based on semantic elements, which represents priority/weight of the segments related to a semantic element or priority/weight information of semantic elements which varies depending on the structure elements.
Moreover, the priority/weight information may be information based on both the semantic elements and the segments. In such case, the link information including the priority/weight information may have additional information describing connection relationships. Furthermore, the priority information may be described as a segment locator for locating a corresponding segment when link information includes the priority/weight information based on the semantic elements. The priority/weight information may also be described as a semantic element locator for locating a corresponding semantic element when the link information includes the priority/weight information based on the segment.
Another embodiment of a multimedia data structure according to the present invention includes link information for connecting a semantic element and a segment wherein the link information includes priority/weight information of the structure element which varies depending on the semantic element, when a multimedia data includes the semantic element structure describing the contents of a multimedia and the segment information structure of the multimedia for browsing the multimedia based on the contents.
A still another embodiment of a multimedia data structure according to the present invention includes link information for connecting a semantic element and a segment wherein the link information includes priority/weight information of the semantic element which varies depending on the structure element, when a multimedia data includes the semantic element structure describing the contents of a multimedia and the segment information structure of the multimedia for browsing the multimedia based on the contents.
A further embodiment of a multimedia data structure according to the present invention includes link information for connecting a semantic element and a segment wherein the link information includes both priority/weight information of the structure element which varies depending on the semantic element and priority/weight information of the semantic element which varies depending on the structure element, when a multimedia data includes the semantic element structure describing the contents of a multimedia and the segment information structure of the multimedia for browsing the multimedia based on content.
A method for browsing a multimedia according to the present invention, includes selecting a semantic element or segment information, selecting a link information of the selected semantic element or segment, and displaying corresponding segment or semantic element information in sequence according to the priority/weight between the corresponding segment element and segment which was obtained by the link information, wherein a multimedia data includes a semantic element structure describing a content of a multimedia and a segment information structure of the multimedia for browsing the multimedia based on content.
Another method for browsing a multimedia according to the present invention, includes assigning xe2x80x98semantic element nxe2x80x99 as a semantic element, selecting xe2x80x98semantic element nxe2x80x99 and reading link information of segments assigned by xe2x80x98semantic element n,xe2x80x99 determining which segments are connected to xe2x80x98semantic element nxe2x80x99 from the link information and reading priority/weight of the connection from the priority information included in the corresponding link information, and displaying the corresponding segments in sequence according to the priority, wherein the priority/weight information based on the semantic element is described by attributes of link information with the corresponding segments.
A further embodiment of a method for browsing a multimedia according to the present invention includes assigning xe2x80x98segment nxe2x80x99 as a structure element, selecting xe2x80x98segment nxe2x80x99 and reading link information of semantic elements assigned by xe2x80x98segment n,xe2x80x99 determining which semantic elements are connected to xe2x80x98segment nxe2x80x99 from the link information and reading priority/weight of the connection from the priority/weight information included in the corresponding link information, and displaying the corresponding semantic elements in sequence according to the priority/weight, wherein priority/weight information based on the structure element described by attributes of link information with the corresponding semantic elements.